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CHAPTER 1 

APPLIED LINEAR ALGEBRA 



1.1 FOUR SPECIAL MATRICES 



An m by n matrix has m rows and n columns and mn entries. We operate on those 
rows and columns to solve linear systems Ax = b and eigenvalue problems Ax = Xx. 
From inputs A and b (and from software like MATLAB) we get outputs x and A. A 
fast stable algorithm is extremely important, and this book includes fast algorithms. 

One purpose of matrices is to store information, but another viewpoint is more 
important for applied mathematics. Often we see the matrix as an "operator." 
A acts on vectors x to produce Ax. The components of x have a meaning — 
displacements or pressures or voltages or prices or concentrations. The operator A 
also has a meaning — in this chapter A takes differences. Then Ax represents pressure 
differences or voltage drops or price differentials. 

Before we turn the problem over to the machine — and also after, when we interpret 
A\b or eig(A) — it is the meaning we want, as well as the numbers. 

This book begins with four special families of matrices — simple and useful, 
absolutely basic. We look first at the properties of these particular matrices K n , C n , 
T n . and B n . (Some properties are obvious, others are hidden.) It is terrific to practice 
linear algebra by working with genuinely important matrices. Here are K2, -K3, K4 in 
the first family, with —1 and 2 and —1 down the diagonals: 



K 2 



2-1 
-1 2 -1 
0-1 2 



K 4 



2 


-1 








1 


2 


-1 








-1 


2 


-1 








-1 


2 



What is significant about K% and K% and K4, and eventually the n by n matrix K n ? 
I will give six answers in the same order that my class gave them — starting with four 
properties of the K's that you can see immediately. 



1 
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1. These matrices are symmetric. The entry in row i, column j also appears 
in row j, column i. Thus Ky = Kji, on opposite sides of the main diagonal. 
Symmetry can be expressed by transposing the whole matrix at once: K = K T . 

2. The matrices K n are sparse. Most of their entries are zero when n gets large. 
K WO o has a million entries, but only 1000 + 999 + 999 are nonzero. 

3. The nonzeros lie in a "band" around the main diagonal, so each K n is banded. 
The band has only three diagonals, so these matrices are tridiagonal. 

Because K is a tridiagonal matrix, Ku = f can be quickly solved. If the unknown 
vector u has a thousand components, we can hnd them in a few thousand steps 
(which take a small fraction of a second). For a full matrix of order n = 1000, solving 
Ku = f would take hundreds of millions of steps. Of course we have to ask if the 
linear equations have a solution in the first place. That question is coming soon. 



4. The matrices have constant diagonals. Right away that property wakes up 
Fourier. It signifies that something is not changing when we move in space or 
time. The problem is shift-invariant or time-invariant. Coefficients are constant. 
The tridiagonal matrix is entirely determined by the three numbers —1, 2, —1. 
These are actually "second difference matrices" but my class never says that. 

The whole world of Fourier transforms is linked to constant-diagonal matrices. In 
signal processing, the matrix D = K/4 is a "highpass filter." Du picks out the rapidly 
varying (high frequency) part of a vector u. It gives a convolution with j(— 1, 2, — 1). 
We use these words to call attention to the Fourier part (Chapter 4) of this book. 

Mathematicians call K a Toeplitz matrix, and MATLAB uses that name: 

The command K = toeplitz([ 2 —1 zeros(l,2)]) constructs from row 1. 

Actually, Fourier will be happier if we make two small changes in K n . Insert — 1 
in the southwest and northeast corners. This completes two diagonals (which circle 
around). All four diagonals of C4 wrap around in this "periodic matrix" or 11 cyclic 
convolution" or circulant matrix: 



Circulant matrix 



C*4 



2-1 

-1 2 -1 

0-1 2 

-1 -1 



-1 


-1 

2 



toeplitz([2 -1 -1]) 



This matrix is singular. It is not invertible. Its determinant is zero. Rather than 
computing that determinant, it is much better to identify a nonzero vector u that 
solves C4U = 0. (If C4 had an inverse, the only solution to C4U = would be 
the zero vector. We could multiply by C^ 1 to find u = 0.) For this matrix, the 
column vector u of all ones (printed as u = (1, 1, 1, 1) with commas) solves C4U = 0. 
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The columns of C add to the zero column. This vector u = ones(4, 1) is in the 
nullspace of C4. The nullspace contains all solutions to Cu = 0. 

Whenever the entries along every row of a matrix add to zero, the matrix is 
certainly singular. The same all-ones vector u is responsible. Matrix multiplication 
Cu adds the column vectors and produces zero. The constant vector u = (1, 1, 1, 1) 
or u = (c, c, c, c) in the nullspace is like the constant C when we integrate a function. 
In calculus, this "arbitrary constant" is not knowable from the derivative. In linear 
algebra, the constant in u = (c, c, c, c) is not knowable from Cu = 0. 

5. All the matrices A = K n are invertible. They are not singular, like C n . There 
is a square matrix A -1 such that A _1 A = I = identity matrix. And if a square 
matrix has an inverse on the left, then also AA -1 = I. This '''inverse matrix" 
is also symmetric when A is symmetric. But A -1 is not sparse. 

Invertibility is not easy to decide from a quick look at a matrix. Theoretically 
one test is to compute the determinant. There is an inverse except when det A = 0, 
because the formula for A -1 includes a division by det A'. But computing the deter- 
minant is almost never done in practice! It is a poor way to hnd u = A -1 /. 

What we actually do is to go ahead with the elimination steps that solve Ku = 
f. Those steps simplify the matrix, to make it triangular. The nonzero pivots on 
the main diagonal of the triangular matrix show that the original A is invertible. 
(Important: We don't want or need A -1 to find u = A -1 /. The inverse would be a 
full matrix, with all positive entries. All we compute is the solution vector u.) 

6. The symmetric matrices K n are positive definite. Those words might be 
new. One goal of Chapter 1 is to explain what this crucial property means 
(A 4 has it, C4 doesn't). Allow me to start by contrasting positive definiteness 
with invertibility, using the words "pivots" and "eigenvalues" that will soon be 
familiar. Please notice the Appendix that summarizes linear algebra. 

{Pivots) An invertible matrix has n nonzero pivots. 

A positive definite symmetric matrix has n positive pivots. 

(Eigenvalues) An invertible matrix has n nonzero eigenvalues. 
A positive definite symmetric matrix has n positive eigenvalues. 

Positive pivots and eigenvalues are tests for positive definiteness, and C4 fails 
those tests because it is singular. Actually C 4 has three positive pivots and 
eigenvalues, so it almost passes. But its fourth eigenvalue is zero (the matrix is 
singular). Since no eigenvalue is negative (A > 0), C4 is positive semidefinite. 

The pivots appear on the main diagonal in Section 1.3, when solving Ku = f 
by elimination. The eigenvalues arise in Kx = Xx. There is also a determinant test 
for positive definiteness (not just det A > 0). The proper definition of a symmetric 
positive definite matrix (it is connected to positive energy) will come in Section 1.6. 
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Changing K n to T n 



After K n and Cn, there are two more families of matrices that you need to know. 
They are symmetric and tridiagonal like the family K n . But the (1,1) entry in T n is 
changed from 2 to 1: 



T n (l,l) = l T 2 



and T-i 



1-1 

-1 2 -1 
0-1 2 



(1) 



That top row (T stands for top) represents a new boundary condition, whose meaning 
we will soon understand. Right now we use T3 as a perfect example of elimination. 
Row operations produce zeros below the diagonal, and the pivots are circled as they 
are found. Two elimination steps reduce T to the upper triangular U. 

Step 1. Add row 1 to row 2, which leaves zeros below the first pivot. 
Step 2. Add the new row 2 to row 3, which produces U. 



T 



© 







Step 1 



© 

-1 



Step 2 







-1 



© 











u. 



All three pivots of T equal 1. We can apply the test for invertibility (three nonzero 
pivots) . T3 also passes the test for positive definiteness (three positive pivots) . In fact 
every T n in this family is positive definite, with all its pivots equal to 1. 

That matrix U has an inverse (which is automatically upper triangular). The 
exceptional fact for this particular U^ 1 is that all upper triangular entries are l's: 



u- 1 



1 -1 
1 -1 
1 



111 

1 1 
1 



triu(ones(3)). 



(2) 



This says that the inverse of a 3 by 3 "difference matrix" is a 3 by 3 "sum matrix." 
This neat inverse of U will lead us to the inverse of T in Problem 2. The product 
U~ X U is the identity matrix I. U takes differences, and U^ 1 takes sums. Taking 
differences and then sums will recover the original vector (111,112,%): 





' 1 




1 







Ui 




til 


~ U2 


Differences from U 







1 


-1 










~ U3 












1 




. U 3 . 




_ U3 


- 






" 1 


1 


1 " 




Ui - 


y-2 




til 


Sums from U -1 







1 


1 




u 2 - 


















1 




u 3 - 
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Changing T n to B n 

The fourth family B n has the last entry also changed from 2 to 1. The 

new boundary condition is being applied at both ends (B stands for both). These 
matrices B n are symmetric and tridiagonal, but you will quickly see that they are 
not invertible. The B n are positive semidefmite but not positive definite: 



B n (n,n) = 1 



Bo 



and B-i 



1 -1 

-1 2 
-1 




-1 
1 



(3) 



Again, elimination brings out the properties of the matrix. The first n — 1 pivots 
will all equal 1, because those rows are not changed from T n . But the change from 2 
to 1 in the last entry of B produces a change from 1 to in the last entry of U: 



B 







1 

1 2 -1 
0-1 1 



1 

Q-l 
0-1 1 



1 

Q-l 




U. (4) 



There are only two pivots. (A pivot must be nonzero.) The last matrix U is certainly 
not invertible. Its determinant is zero, because its third row is all zeros. The constant 



vector 




is 


iii 


the nullspace of U, and therefore it is in the nullspace 


of B 




" 1 


-i 


" 




" 1 " 




" " 




"1-1 " 




" 1 " 




' " 







i - 


1 




1 







and also 


-1 2 -1 




1 




















1 









-1 1 




1 








The whole point of elimination was to simplify a linear system like Bu = 0, without 
changing the solutions. In this case we could have recognized non-invertibility in the 
matrix B, because each row adds to zero. Then the sum of its three columns is the 
zero column. This is what we see when B multiplies the vector (1, 1, 1). 

Let me summarize this section in four lines (all these matrices are symmetric): 



K n and T n are invertible and (more than that) positive definite. 

Cn and B n are singular and (more than that) positive semidefmite. 

The nullspaces of C n and B n contain all the constant vectors 
u = (c, c, . . . , c). Their columns are dependent. 

The nullspaces of K n and T n contain only the zero vector 
u = (0, 0, . . . , 0). Their columns are independent. 
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Matrices in MATLAB 

It is natural to choose MATLAB for linear algebra, but the reader may select another 
system. (Octave is very close, and free. Mathematica and Maple are good for symbolic 
calculation, LAPACK provides excellent codes at no cost in netlib, and there are many 
other linear algebra packages.) We will construct matrices and operate on them in 
the convenient language that MATLAB provides. 

Our first step is to construct the matrices K n . For n = 3, we can enter the 3 by 
3 matrix a row at a time, inside brackets. Rows are separated by a semicolon ; 

K = [ 2 -1 0; -1 2 -1; -1 2] 

For large matrices this is too slow. We can build K s from "eye" and "ones'' : 

eye(8) = 8 by 8 identity matrix ones(7, 1) = column vector of seven l's 

The diagonal part is 2*eye(8). The symbol * means multiplication! The —l's above 
the diagonal of K s have the vector — ones(7, 1) along diagonal 1 of the matrix E: 

Superdiagonal of —l's E = — diag(ones(7, 1), 1) 

The —l's below the diagonal of K s lie on the diagonal numbered —1. For those we 
could change the last argument in E from 1 to —1. Or we can simply transpose E, 
using the all- important symbol E 1 for E T . Then K comes from its three diagonals: 

Tridiagonal matrix K H K = 2* eye(8) + E + E' 

Note: The zeroth diagonal (main diagonal) is the default with no second argument, 
so eye(8)= diag(ones(8,l)). And then diag(eye(8)) = ones(8, 1). 

The constant diagonals make K a Toeplitz matrix. The toeplitz command produces 
K, when each diagonal is determined by a single number 2 or —1 or 0. Use the zeros 
vector for the 6 zeros in the first row of K s : 

Symmetric Toeplitz rowl = [2 —1 zeros(l,6)]; K = toeplitz(rowl) 

For an unsymmetric constant-diagonal matrix, use toeplitz(coll, rowl). Taking coll = 
[1 — 1 ] and rowl = [ 1 ] gives a 4 by 3 backward difference matrix. It has 
two nonzero diagonals, l's and —l's. 

To construct the matrices T and B and C from K, just change entries as in the 
last three lines of this M-file that we have named KTBC.m. Its input is the size n, its 
output is four matrices of that size. The semicolons suppress display of K, T, B, C: 

function [K,T,B,C] = KTBC(n) 

% Create the four special matrices assuming n>l 

K = toeplitz ([2 -1 zeros(l,n-2)]); 

T = K; T(l,l) = 1; 

B = K; B(l,l) = 1; B(n,n) = 1; 

C = K; C(l,n) = -1; C(n,l) = -1; 
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If we happened to want their determinants (we shouldn't!), then with n = 8 
[ det(K) det(T) det(B) det(C) ] produces the output [9 1 0] 

One more point. MATLAB could not store K n as a dense matrix for n = 10,000. 
The 10 8 entries need about 800 megabytes unless we recognize K as sparse. 
The code sparseKTBC.m on the course website avoids storing (and operating on) all 
the zeros. It has K, T, B, or C and n as its first two arguments. The third argument 
is 1 for sparse, for dense (default for narg= 2, no third argument). 

The input to Sparse MATLAB includes the locations of all nonzero entries. The 
command A =sparse(i, j, s, m, n) creates an m by n sparse matrix from the vectors 
i, j, s that list all positions i, j of nonzero entries s. Elimination by \u(A) may produce 
additional nonzeros (called fill-in) which the software will correctly identify. In the 
normal "full" option, zeros are processed like all other numbers. 

It is best to create the list of triplets i, j, s and then call sparse. Insertions A(i, j) = 
s or A(i,j) = A(i,j) + s are more expensive. We return to this point in Section 3.6. 

The sparse KTBC code on the website uses spdiags to enter the three diagonals. 
Here is the toeplitz way to form Kg, all made sparse by its sparse vector start: 

vsp = sparse([2 -1 zeros(l, 6)]) % please look at each output 

Ksp= toeplitz(vsp) % sparse format gives the nonzero positions and entries 

bsp = Ksp(:, 2) % colon keeps all rows of column 2, so bsp = column 2 of Ksp 

usp = Ksp\bsp % zeros in Ksp and bsp are not processed, solution: usp(2) = 1 

uuu= full(usp) % return from sparse format to the full uuu = [0 100000 0] 

Note The open source language Python is also very attractive and convenient. 

The next sections will use all four matrices in these basic tasks of linear algebra: 

(1.2) The finite difference matrices K, T, B, C include boundary conditions 

(1.3) Elimination produces pivots in D and triangular factors in LDL T 

(1.4) Point loads produce inverse matrices K^ 1 and T _1 

(1.5) The eigenvalues and eigenvectors of K, T, B, C involve sines and cosines. 

You will see K\f in 1.2, lu(K) in 1.3, inv(K) in 1.4, eig(K) in 1.5, and chol(K) in 1.6. 

I very much hope that you will come to know and like these special matrices. 
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■ WORKED EXAMPLES ■ 



1.1 A Bu = f and Cu = f might be solvable even though B and C are singular ! 

Show that every vector / = Bu has fx + / 2 + • • • + f n = 0. Physical meaning: 
the external forces balance. Linear algebra meaning: Bu = f is solvable when / 
is perpendicular to the all-ones column vector e = (1, 1, 1, 1, . . . ) = ones(n, 1). 



Solution Bu is a vector of "differences" of u's. Those differences always add to zero: 



f = Bu = 



1 


-1 








Ui 




Ui 


- u 2 




1 


2 


-1 






u 2 




-Ul 


f 2u 2 - 


"3 




-1 


2 


-1 




u 3 






f 2u 3 - 








-1 


1 




u^ 






~«3 + 





All terms cancel in [u\ — u 2 ) + (— U\ + 2u 2 — u :i ) + (—u 2 + 2u 3 — u 4 ) + (—u 3 + 114) = 0. 
The dot product with e = (1, 1, 1, 1) is that sum / T e = fi + f 2 + / 3 + f\ = 0. 

Dot product / • e = / T e = f iei + f 2 e 2 + / 3 e 3 + / 4 e 4 (f'*e in MATLAB). 



A second explanation for / T e = starts from the fact that Be = 0. The all-ones 
vector e is in the nullspace of B. Transposing / = Bu gives / T = u T B T , since the 
transpose of a product has the individual transposes in reverse order. This 
matrix B is symmetric so B T = B. Then 

f T e = u T B T e is equal to u T Be = u T = 0. 



Conclusion Bu = / is only solvable when / is perpendicular to the all-ones vector 
e. (The same is true for Cu = f. Again the differences cancel out.) The external 
forces balance when the f's add to zero. The command B\f will produce Inf because 
B is square and singular, but the "pseudoinverse" u = pinv(B) * f will succeed. (Or 
add a zero row to B and / before the command B\f, to make the system rectangular.) 



1.1 B The "fixed- free" matrix H changes the last entry of K from 2 to 1. Connect 
H to the "free-fixed" T (first entry = 1) by using the reverse identity matrix J: 



H 



2 -1 
-1 2 
-1 



comes from 

JTJ via the J 

reverse identity J 



1 

1 

1 



Chapter 2 shows how T comes from a tower structure (free at the top). H comes 
from a hanging structure (free at the bottom). Two MATLAB constructions are 



H= toeplitz([2 -1 0]); #(3,3) = 1 or J = fliplr(eye(3)); H = J *T * J 
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Solution JT reverses the rows of T. Then JTJ reverses the columns to give H: 



1-1 

-1 2 -1 
0-1 2 



JT 



rows 




-1 
1 



-1 2 
2 -1 
-1 



(JT)J = H . 
(columns too) 



We could reverse columns first by TJ. Then J(TJ) would be the same matrix H as 
(JT) J. The parentheses never matter in (AB)C = A(BC) ! 

Any permutation matrix like J has the rows of the identity matrix / in some 
order. There are six 3 by 3 permutation matrices because there are six orders for 
the numbers 1,2,3. The inverse of every permutation matrix is its transpose. This 
particular J is symmetric, so it has J = J T = J -1 as you can easily check: 



H = JTJ so H- 



J-'T-'J- 1 



which is H 



JT-\J. 



(5) 



With back = 3: — 1:1, reordering to JTJ is H = T(back, back) in MATLAB. 

Problem Set 1.1 

Problems 1—4 are about T _1 and Problems 5—8 are about K —1 . 



The inverses of T3 and T4 (with T u = 1 in the top corner) are 



T7 



1 1 1 



and Ta 



1111 



Guess T 5 1 and multiply by T 5 . Find a simple formula for the entries of T n 1 on 
and below the diagonal (i > j), and then on and above the diagonal (i < j). 



Compute T :i 1 in three steps, using U and U 1 in equation (2) 
1 



l's 



Check that T 3 = U U, where U has l's on the main diagonal and 
along the diagonal above. Its transpose U T is lower triangular. 

Check that UU^ 1 = I when U^ 1 has l's on and above the main diagonal 

Invert U T U to find T^ 1 = (?7~ 1 )(£/~ 1 ) T . Inverses come in reverse orderl 



The difference matrix U = U 5 in MATLAB is eye(5)-diag(ones(4,l),l). Con- 
struct the sum matrix S from triu(ones(5)). (This keeps the upper triangular 
part of the 5 by 5 all-ones matrix.) Multiply U * S to verify that S = U^ 1 . 

For every n, Sn = UnT 1 is upper triangular with ones on and above the diagonal. 
For n = 4 check that SS T produces the matrix T^T 1 predicted in Problem 1. 
Why is SS T certain to be a symmetric matrix? 
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The inverses of K 5 and K 4 (please also invert K 2 ) have fractions 



1 
det 



1 1 

4'5 : 



k* 1 



3 2 1 
2 4 2 
1 2 3 



and K 4 1 



4 3 2 1 

3 6 4 2 

2 4 6 3 

12 3 4 



First guess the determinant of K = K 5 . Then compute det(K) and \n\/(K) and 
det(A')* inv(iT) — any software is allowed. 

(Challenge problem) Find a formula for the i, j entry of K^ 1 below the diagonal 
(i > j)- Those entries grow linearly along every row and up every column. 
(Section 1.4 will come back to these important inverses.) Problem 7 below is 
developed in the Worked Example of Section 1.4. 



A column u times a row v T is a rank-one matrix uv 

multiples of u, and all rows are multiples of v T . T 4 _1 — K^T 1 



T All columns are 
has rank 1: 



T -i 





" 16 


12 


8 


4 " 




" 4 " 


1 


12 


9 


6 


3 


1 


3 


5 


8 


6 


4 


2 


~ 5 


2 




4 


3 


2 


1 




1 



[4 3 2 1] 



Write K$ — T% in this special form uv T . Predict a similar formula for T 3 1 — K 3 . 

8 (a) Based on Problem 7, predict the i, j entry of T 5 -1 — K^ 1 below the diagonal. 

(b) Subtract this from your answer to Problem 1 (the formula for T^ 1 when 
i > j)- This gives the not-so-simple formula for K^ 1 . 

9 Following Example 1.1 A with C instead of B, show that e = (1, 1, 1, 1) is 
perpendicular to each column of C4. Solve Cu = f = (1,-1, 1,-1) with the 
singular matrix C by u = pinv(C) * f. Try u = C\e and C\f, before and after 
adding a fifth equation = 0. 

10 The "hanging matrix" H in Worked Example 1.1 B changes the last entry of K 3 
to H33 = 1. Find the inverse matrix from H^ 1 = JT~ X J. Find the inverse also 
from H = UU 1 - (check upper times lower triangular!) and H^ 1 = ((7 _1 ) T ?7 _1 . 

11 Suppose U is any upper triangular matrix and J is the reverse identity matrix 
in 1.1 B. Then JU is a "southeast matrix". What geographies are UJ and 
JUJ? By experiment, a southeast matrix times a northwest matrix is . 

12 Carry out elimination on the 4 by 4 circulant matrix C 4 to reach an upper 
triangular U (or try [L, U] = \u(C) in MATLAB). Two points to notice: The 

last entry of U is because C is singular. The last column of U has new 

nonzeros. Explain why this "fill-in" happens. 
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13 By hand, can you factor the circulant C4 (with three nonzero diagonals, allowing 
wraparound) into circulants L times U (with two nonzero diagonals, allowing 
wraparound so not truly triangular)? 

14 Gradually reduce the diagonal 2, 2, 2 in the matrix K 3 until you reach a singular 

matrix M. This happens when the diagonal entries reach . Check the 

determinant as you go, and find a nonzero vector that solves Mu = 0. 

Questions 15-21 bring out important facts about matrix multiplication. 

15 How many individual multiplications to create Ax and A 2 and AB? 

An x n x n x 1 x n x n A m x n B n x p = {AB) mxp 



16 You can multiply Ax by rows (the usual way) or by columns (more important). 
Do this multiplication both ways: 



By rows 
By columns 



"2 3" 




"l" 


4 5 




to 


"2 3" 




T 


4 5^ 




to 

1 



inner product using row 1 
inner product using row 2 



1 



combination 
of columns 



17 The product Ax is a linear combination of the columns of A. The equations 
Ax = b have a solution vector x exactly when b is a of the columns. 

Give an example in which b is not in the column space of A. There is no solution 
to Ax = b, because b is not a combination of the columns of A. 

18 Compute C = AB by multiplying the matrix A times each column of B: 



19 



"2 


00 




"1 


2" 




" 8 *" 


4 


5 




2 


4 




14 * 



Thus, A * B(:,j) = C(:,j). 

You can also compute AB by multiplying each row of A times B: 



"2 3" 




"1 2" 




2 * row 1 + 3 * row 2 




"8 16 


4 5 




2 4 




4 * row 1 + 5 * row 2 




* * 



A solution to Bx = is also a solution to (AB)x = 0. Why? From 



Bx 



"1 


l\2 




"-2" 




'0" 


2 


4 




1 








how do we know ABx 



"8 16" 




"-2" 




"0" 


* * 




1 
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20 The four ways to find AB give numbers, columns, rows, and matrices: 



1 (rows of A) times (columns of B) 

2 A times (columns of B) 

3 (rows of A) times B 

4 (columns of A) times (rows of B) 



C(i,j) = A(i,:)*B(:,j) 
C(:,j)=A* B(:J) 
C(i,:) = A(i,:)*B 

for k=l:n, C = C + A(:,k) * B(k,:); 
end 



Finish these 8 multiplications for columns times rows. How many for n by ril 



2 3 
4 5 



1 2 

2 4 



T 21 



[2 4] 



2 4 




* 


* 




8 




4 8 


+ 


* 


* 




* 


* 



21 Which one of these equations is true for all n by n matrices A and B1 

AB = BA (AB)A = A(BA) (AB)B = B(BA) (AB) 2 = A 2 B 2 . 



22 Use n = 1000; e = ones(n, 1); K = spdiags([— e, 2 * e, — e], 
^1000 as a sparse matrix. Solve the sparse equation Ku 
the solution by plot(u). 



-1 : 1, n, n); to enter 
e by u = K\e. Plot 



23 Create 4-component vectors u, v, w and enter A = spdiags([u, v, w], —1 : 1,4, 4). 
Which components of u and w are left out from the —1 and 1 diagonals of A? 

24 Build the sparse identity matrix / = sparse(i, j, s, 100, 100) by creating vectors 
i, j, s of positions i, j with nonzero entries s. (You could use a for loop.) In this 
case speye(100) is quicker. Notice that sparse(eye(10000)) would be a disaster, 
since there isn't room to store eye( 10000) before making it sparse. 

25 The only solution to Ku = or Tu = is u = 0, so K and T are invertible. For 



proof, suppose Ui is the largest component of u. If 



2uj 



this forces u, 



Mj+i. Then the next equations force every Uj 



u i+ i is zero, 
At 
0. 



the end, when the boundary is reached, — u„_j + 2u n only gives zero if u 
Why does this "diagonally dominant" argument fail for B and C? 

26 For which vectors v is toeplitz(w) a circulant matrix (cyclic diagonals) ? 

27 (Important) Show that the 3 by 3 matrix K comes from Ajylo: 



A 



-110 
0-110 
0-11 



is a "difference matrix" 



Which column of A would you remove to produce Ai with T = AjAi ? Which 
column would you remove next to produce A^ with B = A^A2 ? The differ- 
ence matrices A ,Ai,A 2 have 0,1,2 boundary conditions. So do the "second 
differences" T, and B. 



